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Constructive  Approximation  by 
(V,f)-Reproducing  Kernels 


Mohammed-Najib  Benbourhim 


Abstract.  In  this  paper  we  propose  a constructive  method  to  build 
reproducing  kernels.  We  define  the  notion  of  (V,  /"(-reproducing  kernel, 
and  prove  that  every  reproducing  kernel  is  a (V,  /(-reproducing  kernel. 
We  study  the  minimal  approximation  by  these  (V,  /) -reproducing  kernels 
for  different  choices  of  V and  /.  Examples  to  which  our  results  apply 
include  curve  and  surface  fitting. 


§1.  (V,f)-Reproducing  Kernels 

For  any  set  (respectively  locally  compact  set)  fl,  we  denote  by  ]Rn  (respectively 
Cm(fi))  the  space  of  real-valued  functions  (respectively  m-times  continuously 
differentiable  functions)  defined  on  fl  equipped  with  the  topology  of  pointwise 
convergence  (respectively  uniform  convergence  on  the  compact  subsets  of  ft) . 
Let  us  recall  some  definitions. 

Definition  1.1.  A real-valued  function  H defined  on  fl  x fl  is  a reproducing 
kernel  on  fl  x fl  if 

1)  H is  symmetric:  H(t,s)  = H(s,t)  for  all  t,  s € fl, 

2)  H is  of  positive  type: 

k,l=N 

Y,  \hH(tk,ti)  > 0, 

fc,I=l 

for  any  Unite  point  set  offl  and  real  numbers  {^k)kLi- 

Definition  1.2.  A vector  subspace  ft  of  Rn  is  said  to  be  a hilbertian  subspace 
of  ]Rn  (respectively  Cm(fl) ) if 

1)  H is  a Hilbert  space, 

2)  The  natural  injection  from  ft  into  Rn  (respectively  Cm(fl) ) is  continuous. 

We  review  some  important  results  on  reproducing  kernels  which  are  studied 
in  [4], 
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Theorem  1.1. 

1)  A Hilbert  space  H ( respectively  a real-valued  function  H defined  on  fix  fl) 
is  a hilbertian  subspace  of  (respectively  a reproducing  kernel  onflxfl) 
if  and  only  if  there  exists  one  and  only  one  reproducing  kernel  H on  fix  fl 
(respectively  hilbertian  subspace  7 i of  such  that 

u(t)  = (u\H(-,t))H,  VfGfi,  VueH. 

ft  is  called  the  hilbertian  subspace  associated  with  H . 

2)  For  any  hilbertian  basis  (fi)i£i  of  ft:  H(t,s ) = 

iei 

3)  If  H is  separately  m-times  continuously  differentiable,  then  ft  is  a hilber- 
tian subspace  of  Cm(fl). 

4)  The  vector  space  ft0  =span{(H(-,  is  dense  in  ft. 

Let  (V,  {■  | • )y ) be  a Hilbert  space,  ftbea  set  and  / be  a function  from  fl  into 

V.  ' ' 

Definition  1.3.  For  all  f : fl  ■ — > V,  we  define  a (V,  /)-reproducing  kernel  Hf 

by 

Hf(t,s)  = {f(t)  | f(s))v,  V(t,s)  €flxfl.  (1.1) 


We  have  the  following  result: 

Theorem  1.2.  Hf  defined  by  (1.1)  is  a reproducing  kernel  on  fl  x fl  and  its 
associated  hilbertian  subspace  fif  of  Hn  is 

7 if  = jtu  G Rn  | 3u  € V : w(t)  = ( u \ f(t))v , Vf  6 ftj. 


Proof:  One  can  easily  verify  that  Hf  is  a reproducing  kernel. 

Let  Hf  : V — > ]Rn  be  defined  by  ( Hfu)(t ) = ( u \ f(t))v ■ The  mapping  Hj 
is  linear,  and  the  inequality 

I ( Hsu)(t ) | < | u \v  | /(<)  |K=|  u \v  Hf{t,t)\ 

for  all  t G 0 and  for  all  u € V implies  that  it  is  continuous.  Let  M.  be  the 
closure  in  V of  the  vector  space  span {(/(f))t€n),  and  Pm  the  orthogonal 
projector  on  M.  We  define  on  Hf  = Hf{V)  the  bilinear  form 

(Hfu  | Hfv)nf  = { PMu  | PMv)v- 

It  is  easy  to  see  that  this  form  is  a scalar  product  on  Hf.  Then  the  linear 
mapping  Hf  : M — » H;  is  an  isometry,  and  consequently  (Hf,  (■  \ ■)hs)  is  a 
Hilbert  space.  For  all  t G fl,  the  function 
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Hf{t,  •)  : s G ft  — * Hf(t,  s)  = ( f(t ) | f(s))v, 
is  an  element  of  Hf , and  satisfies  the  reproducing  formula 

(Hfu)(t)  = {Hfu\Hf(t,  ■))«„  V«€V. 

Consequently  (see  Theorem  1.1),  Hf  is  a hilbertian  subspace  of  1RQ  and  admits 
Hf  as  reproducing  kernel.  □ 

Theorem  1.3.  Let  oj  be  a set  and  c a mapping  from  oj  to  ft.  Then 


(Hc)f(y,z)  = Hf(c(y),c(z))  = (f(c(y))  | f(c(z)))v 
is  a reproducing  kernel  on  u>  x u>. 

Proof:  For  all  y £ oj,  f(c(y))  is  in  V.  The  function  (He) j is  symmetric  and 
is  of  positive  type: 

k,l=N  k=N  k—N 

Y AfcA i(Hc)f(yk,yi)  = (Y  A kf(c(yk))  \ Y xkf(c(yk)))v  > 0.  □ 

k,l= 1 k=l  k= 1 


Example  1.1.  Let  V — L2(a,b),  ft  a subset  of  1R  and  f(t)(x ) = exp  (cad) 
where  c is  a real  constant.  Then 

„ . f (exp(c6(t  + s))  - exp(ca(t  + s)))/(c(t  + s)),  if  ( t + s)^0, 

Hf(t,s)  = < 

lb  — a,  otherwise. 

Example  1.2.  Let  V = L2(1R+),  and  suppose  ft  is  a subset  of  ft". 

For  all  functions  c : ft  — > (0,  +oo),  we  have 
(i)  If  f(t)(x)  — -r  exp-^*^*!2 , then  Hf(t,s) 


y/c(ty+~c(s) 

, and  in  particular  if 


(ii)  If  f(t)(x)  = exp  c^x,  then  Hf(t,s)  = - * 

c(t ) + c(s) 

p(t ) 

c(t)  = ----  (with  P(t)  and  Q(t)  polynomials),  we  obtain  the  rational  repro- 

Q(t) 

ducing  kernel 

Q(t)Q(s) 


Hf(t,  s)  = 


P(t)Q(s)  + P(s)Q(t) 


§2.  (V,f)-Reproducing  Kernels  of  Convolution  Type 


We  consider  the  case  where 

1)  y = L2(Hn)  and  ft  = IT. 

2)  f(t)(x)  = f(t  — x),  with  / in  the  familar  Sobolev  space  Lfm(lRn). 
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Theorem  2.1.  We  have  the  following  properties: 

1)  Hf(t,s)  = hf(t  - s)  with  hf(0  = (f  * /)(£)  = T{\  Tf  |2)(0,  where 
f(x)  — f(—x)  and  Tf  is  the  Fourier  transform  of  f. 

2)  hf  6 C™(]Rn)  = \u  e Cm(Hn)  | lim  {Dau)(t)  = 0,  0 <|  a |<  m)  . 

v |t|— *oo  J 

3)  The  associated  hilbertian  subspace  of  Hj  is 

Tif  = f * L2(]Rn)  •— > C™(Rn)  ( continuous  embedding) . 

4)  In  particular,  if  | Tf  |>  0,  then 

-Hf  = {W  € 5'  I Tw  e Lloc(Un),  e h2(Rn)}, 
equipped  with  the  scalar  product 

5)  If  f is  radial,  then  hj  is  radial:  Hf(t,  s)  = h/(|  t - s |). 

6)  For  all  distinct  points  {tk}k=i  hi  R",  the  matrix  II fr  = (llj(tk,ti))  ]<k  1<N 
is  invertible  (strictly  positive  deGnite). 

Proof: 

1)  We  have 

Hf(t,  s)  = (2»)-»  / | Tf(0\2d(  = T{\  Tf  |2)(t  - s). 

J R" 

2)  / € Hm{l Rn)  =>  = (£>“/)  * / e C0°(Rn)  for  0 <|  q |<  m,  (see  [2]). 

3)  is  a consequence  of  Theorem  1.2  and  the  property  given  in  1). 

4)  Since  H;  *— > Col(]Rn)  > S , we  have  the  equivalences: 

{ w € Hf}  44  {3u  € L2(Rn)  : Tw  = TuTf } 

4^  {Pm  6 LL(Rn),  yj  € L2(R")}- 

From  Theorem  1.2,  we  have 

w(t)  = (u  | f(t  - -))l2( R")  = (-Pm^  I /(<  - -))l2( R")  = (fM«  * /)(<)• 

Then  Tw  = TPm  uTf,  and 

(u>l  | W2 )nf  = / PMU\{x)PMU2{x)dx 

J R” 

= f TPMUl(0  TPMu2(f)d(  = [ 

J R"  JR"  I I 


(V,f) -Reproducing  Kernels 
5)  For  any  orthogonal  matrix  A, 
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hf(At)  = f f(x)f(At  — x)dx  = f f(Ax)f(A(t-x))dx 
J Rn  JR" 

= f f{x)f(t-x)dx  = hf(t), 

J R" 


since  f{Ax)  = f(x)  and  | detA  |=  1. 

6)  We  suppose  that  / ^ 0 in  L2(lRn).  Since  the  matrix 

Hn  = ( / f(tk  — x)f(ti  — x)dx  ) is  a Gram  matrix,  it  is  invertible 

RT.  Jl  <k,l<N 

if  and  only  if  the  system  {f(tk  — -)}^=i  is  lineary  independent  in  L2(lRn).  If 
k—N 

ckf(tk  — x)  = 0 in  L2(]Rn),  for  ck  € H,  1 < k < N,  then  by  the  Fourier 

fc=i 

k=N 

transform  we  get  (£  cke~iM  ())?m  = 0 in  L2(lRn). 


k=N 


The  Lebesgue  measure  of  the  set  M = {(,  £ IR”  | ^ cke  = 0}  is  equal 


fc=i 


to  zero.  Then  Tf  vanishes  outside  N , i.e:  T f = 0 in  L2(Rn)  and  by  the 
inverse  Fourier  transform,  f = 0 in  L2(1R"),  which  complete  the  proof.  □ 


Example  2.1.  Let  u(x)  = (1—  | x |)+  and  Tu  — v.  We  have  u(a;)  = 
(i)  Taking  f = T{\u  |5),  Hf(t,s)  = (1-  1 1 - s |)+. 


sm2(|) 


(ii)  Taking  / = T{\  v | 5 ),  Hf(t,  s ) 


^2(¥) 
(t-s)2  ’ 


Example  2.2.  ( Bessel  reproducing  kernels)  For  n € IN,  a £ R and  a > n; 
consider  Ga  € L2( R”)  defined  by  T{Ga)(x)  = (1  + | x |2)_t . 

(i)  Taking  / = T{\  Ga  |*),  Hf(t,s)  = (1+  1 1 - s |2)-* . 

(ii)  Taking  / = T(\  FGn+i  |5),  Hf(t,s)  = y exp(-|  t - s |)  and 

Kf  = H (IR")  (Sobolev  space). 


Example  2.3.  (t/-B-spline  reproducing  kernels)  Let 

1)  Yi(x)  = px+- 

2)  v £ S'  (distributions  with  compact  support)  such  that  v(p)  = 0,  for  all 
polynomial  p in  'Pi(lR). 

3)  / = v*Yi. 

For  such  functions  /,  we  give  the  following  theorem  without  proof. 
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Theorem  2.2.  We  have: 

1)  / e L2(] R). 

2)  For  all  u in  F,+1(1R)  = {v  £ F?oc(lR)/  w<i+1>  G L2(R)}  (Beppo-Levi 
space)  we  have  / u^l+1\x)f(t  — x)dx  = (u  * u)(t). 

Jr 

3)  Hf(t,  s)  = (— 1)'(£>  * i/  * y2l+i)(t  - a)  and  H;  = v*  Vl+1(1R). 

In  the  particular  case  of  divided  differences,  v is  defined  as  the  mth-iterated 
convolution  v = , and  v = 

(o— a)  ’ (b— a) 


§3.  Data  Fitting  by  (V,f)-Reproducing  Kernels 


Let  {*fc}£Li 
7if  into  Rw 


a set  of  distinct  points  in  f2,  and  define  a linear  operator  v4jv  from 
by  An{u)  = (u(tk))liksN. 


Definition  3.1.  For  all  z n £ RA  and  e £ [0, 1)  we  define  a spline  to  be  any 
solution  of  the  following  minimal  approximation  problem: 


(AM)  : 

inf  ((1  - 
u€C<  V 

«){«  1 u)n,  + - ZnIIrn) 

where 

Ct  = ■ 

\ Mm, 

if  e = 0 ( Interpolation ), 

l«/. 

if  e €]0, 1[  ( Smoothing ). 

The  following  theorem  gives  the  spline  in  the  case  e ^ 0. 

Theorem  3.1.  For  all  ( e,z n)  e]0,  l[xIR;V,  the  problem  Pe(zjv ) (Smoothing) 
admits  a unique  solution 


k=N 


fc  = 1 


where  the  coefficients  *Ae  = (A), . . . , X(N)  £ It A are  the  solution  of  the  system 


(Hn  + _ -frv) A/v  = zn , 


with  Hn  — (H f(tk,ti))i<k,i<N  and  In  is  the  identity  matrix. 

Proof:  1)  From  the  continuous  embedding:  Tif  c— » Rn  (see  Theorem  1.2), 
we  deduce  that  An  is  continuous.  2)  AnCHj)  is  closed  as  a vector  subspace 
of  R;V . Then  from  the  general  spline  theory  (see  [1,3])  we  get  the  theorem.  □ 


(VJ)-Reproducing  Kernels 
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Theorem  3.2.  The  following  two  properties  are  equivalent: 

1)  For  all  zjv  £ 1R‘V,  the  problem  Po(zjv)  admits  a unique  solution. 

2)  The  system  {/(tkJj^Li  1S  linearly  independent  in  V. 

Proof:  For  all  zjv  € IR'V,  the  problem  Po(zjv)  admits  a unique  solution  if  and 
only  if  the  matrix  H n — (Hj(tk,  ti))1<k  1<N  is  invertible  (see  [1,3]).  Since  the 

matrix  Hn  = (( f(tk ) | f(ti))v)1<k  i<N  is  a Gram  matrix,  it  is  invertible  if 
and  only  if  the  system  {f(tk)}k=1  is  lineary  independent  in  V.  □ 

Furthermore,  for  the  particular  case  V = L2(IRn)  and  / € Pm(lRn), 
Theorem  3.2  and  the  property  (6)  of  Theorem  2.1  imply  the  following  theorem. 

Theorem  3.3.  For  all  f € Hm(lRn)  and  zjv  £ R^,  the  problem  Pq(zn) 
(Interpolation ) admits  a unique  solution 

*°(t)=  ^ X°kHf(t,tk), 

k= 1 

where  the  coefficients  tA°  = (A®, . . . , \°N)  £ 1R'V  are  the  solution  of  the  system 

HnA°n  = zjv, 

with  Hn  = ( Hf(tk,ti))i<kj<N  and  In  is  the  identity  matrix. 

§4.  Data  Fitting  Preserving  Polynomials 

Let  Pd(Rn)  the  vector  space  of  polynomials  of  degree  at  most  d.  We  suppose: 
(HI)  For  all  p £ P<j(lRn)  the  subset  of  HI"  is  such  that 

{p{tk)  =0,  1 < k < N}  •<=>•  p = 0. 

(H2)  H/nPd(Rn)  = {0}. 

We  remark  that  in  the  case  V = L2(IRn)  and  / € Hm( Rn),  the  hypothesis 
(H2)  is  satisfied  because  Hf  C Cg^lR")  (see  Theorem  2.1(2)),  and 

C(R”)  H Vd(l R")  = {0}. 

Let  Hj  be  the  Hilbert  direct  sum:  H'j  = Tif  ©P<j(]R").  We  denote  by  n j 
the  orthogonal  projector  from  H'j  onto  Hf,  and  we  define  on  Hj  the  linear 
mapping  AN(u)  = (u(tk))1<k<N  € RW.  For  all  (e,zN)  £ [0,1]  x Rw,  we 
consider  the  following  minimal  approximation  problem  in  Hj : 

(p<:(zn))  : jnf  ((l-e)(n/(w)  | n^u))*,  + e\\ANu  - zat||rw), 

where 

!A~n{zn},  if  e = 0 (Interpolation), 

Hf,  if  e £]0, 1[  (Smoothing), 

Pcj(IRn),  if  e = 1 (Least  squares). 

The  hypothesis  (HI)  implies  that  the  problem  Pi(zjv)  admits  a unique  solu- 
tion. In  the  case  e €]0, 1[  (Smoothing)  we  have  the  following  theorem: 
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Theorem  4.1.  For  all  (e, z/v)  £]0,  lfxR^,  the  problem  P£(zjv)  admits  a 
unique  solution  a*.  In  the  case  e ejO,  1[,  the  solution  o'  is  given  by 

k=N  i=nd 

^(t)  = xkHf(t,tk)  + KPitt), 

k=l  i=l 

where  the  coefficients  tht  — (Af, . . . , A^)  G 1RN  a ndtBc  = (b[, . . . , b„d)  G R"" 
are  the  solution  of  the  system 

(Hn  + ^In  E\(A‘\_(zn\ 

V 4F  0 J \BeJ  \0  )' 

with 

1)  = (Hj{tk,ti))l<k  1<N  and  In  is  the  identity  matrix, 

2)  E = with  Ekii  = Pi(tk)  and  {pi)i<i<nd  is  a basis  ofVd{ R”). 

In  particular,  if  there  exists  p G Fd(Rn)  such  that  {p(tk)  = 1 < k < N}, 

then  oc  — p (preserving  polynomials  property). 

Proof:  Theorem  4.1  is  a consequence  of  general  spline  theory  (see  [1,3]): 

1)  An  is  continuous  since  Hj  is  a hilbertian  subspace  of  Rn. 

2)  11/  is  continuous  and  Hf(Hj)  = Hf  is  closed  since  11/  is  an  orthogonal 
projector. 

3)  ker  Aw  flkerll/  = {0}:  derives  from  the  hypothesis  (HI)  and  the  fact 
that  kerll/  = Pd(Rn). 

4)  ker  An  + ker  Hf  is  closed  since  ker  Hf  is  a finite  dimensional  vector  space. 
□ 
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